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Applicants have read and considered the Office Action dated February 4, 201 1 and the 
references cited therein. Claims 1, 28, 30, 33, 36, 37 and 40 have now been amended. Claims 4- 
5, 14, 26-27, 32, 35 and 39 have been cancelled without prejudice or disclaimer. Claims 1-3, 6- 
13, 15-25, 28-31, 33-34, 36-38 and 40-42 are currently pending. Reconsideration is hereby 
requested. 

In the Action, claims 1, 3, 9-10, 12-15, 17-20, 23-32, 37-39 and 41-42 were rejected 
under 35 U.S.C. § 103(a) as being unpatentable over Takahashi (US 5,522,789) in view of 
Suzuki (US 5,796,427). Claims 2, 4-5 and 33-35 were rejected under 35 U.S.C. § 103(a) as 
being unpatentable over Takahashi and Suzuki in view of Adelson (US 5,076,687). Claim 36 
was rejected under 35 U.S.C. § 103(as) over the combination of Takahashi, Suzuki and Adelson 
and further in view of Watanabe (US 5,812,187). Claim 40 was rejected under 35 U.S.C. § 
103(a) as being unpatentable over Takahashi, Suzuki and further in view of Watanabe. Claims 
6-8 and 1 1 were rejected under 35 U.S.C. § 103(a) as being unpatentable over Takahashi and 
Suzuki in view of Watanabe. Finally, claims 16 and 21-22 were rejected under 35 U.S.C. § 
103(a) as being unpatentable over Takahashi and Suzuki in view of Watanabe. Applicants 
respectfully traverse the rejections. 

The Invention 

The present invention relates to a stereoscopic endoscopic system and method for 
producing stereoscopic color images of an object while compensating for movements of the 
image sensor assembly (e.g., as a result of human operator hand movements). The system 
includes a sensor assembly, a movement detector and a processing unit all coupled there 
between. The sensor assembly acquires stereoscopic images of an object. The movement 
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detector detects the movements of the sensor assembly. The processing unit receives the 
detected stereoscopic images from the sensor assembly and receives the sensor movement data 
from the movement detector. The processing unit determines a location of origin which 
corresponds to the average of the movements of the sensor assembly (e.g., point Oi of Figures 
25A-25F corresponds to the location of origin for the respective images). The processing unit 
determines the distance of each of the sensor locations corresponding to each of the detected 
images, from the location of origin. The processing unit determines, according to the direction 
and the distance of each sensor location from the location of origin, a sub-matrix of the 
corresponding detected image. For example, each of the sub-matrices is selected such that the 
center point thereof corresponds to the location of origin. In this manner, each of the plurality of 
selected sub-matrices corresponds to the same image area and a stereoscopic image of the object 
is produced while compensating for sensor movements. 

(2) Arguments 

Takahashi 

Takahashi describes a stereoscopic endoscopic imaging apparatus. The system of 
Takahashi includes a pair of relay lens systems, an imagery optical system and a pair of imaging 
devices. The imagery optical system is a verifocal optical system. The system detects 
movements of elements within the verifocal optical imagery system and accordingly moves the 
image detectors. 

The claimed invention is directed at producing stereoscopic endoscopic images while 
compensating for movements of the sensor assembly employed for acquiring images of an 
object. Takahashi fails to disclose or discuss compensating for the movements of the sensor 
assembly (i.e., the sensor assembly is positioned at a slightly different position when acquiring 
each of the images, as a result of, for example, the movements of the hand of the operator). 

In particular, according to the claimed invention the average of the movements of the 
sensor assembly must be constant, and the average movement corresponds to a location of origin 
(i.e., the average of movements with respect to the location of origin). Takahashi, as mentioned 
above, does not discuss movements of the sensor assembly, in general, and does not discuss 
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those movements being average over a location of origin. 

In the device and method recited in the claims of the present application, the location of 
origin marks the center point of each of the associated sub-matrices (i.e., each sub-matrix is 
selected such that the center point thereof is positioned at the location of origin). Takahashi does 
not suggest selecting sub-matrices out of the detected images for compensating for sensor 
assembly movements. 

Suzuki 

Suzuki describes an apparatus for acquiring images while correcting for image fluctuations 
resulting from movements of the camera. The apparatus of Suzuki includes a fluctuation 
detector, a memory controller and a field memory. The memory controller is coupled between 
the fluctuation detector and the memory field. An image detector (the image detector is not a 
part of the system of Suzuki) detects two images of an object. The fluctuation detector detects a 
movement signal according to the movement of the image detector from the position of the 
image detector when acquiring the first image to the position when acquiring the second image. 
The memory controller writes the first image detected by the image detector into the memory 
field. The memory controller writes a portion of the second image, corresponding to the 
common image area of the first image and the second image, to the field memory. The memory 
controller determines the common image area according to the detected movement signal. In 
this manner, the system of Suzuki compensates for image detector fluctuations between 
consecutive images. 

The system and method of the present invention require that a location of origin is defined 
for a plurality of detected images. The location of origin is defined according to the average of 
the movements of the image sensor assembly. Suzuki, on the other hand describes detecting the 
fluctuations in the position of the image detector when acquiring two consecutive images. 
Suzuki does not describe or suggest determining the average of such fluctuations over a plurality 
of acquired images. Therefore, Suzuki does not describe determining a location of origin 
according to the average of the detected movements of the images detector. 

The present invention describes selecting a sub-matrix out of each detected image. The 
sub-matrix corresponds to the determined location of origin. The center point of each of the sub- 
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matrices is positioned at the location of origin. In this manner, the system compensates for 
movements of the image sensor assembly occurring while acquiring a plurality of images. The 
present invention describes producing stereoscopic color images from the plurality of sub- 
matrices. It is noted that the size of each of the sub-matrices is similar and is determined 
according to the detected movements of the image sensor assembly. 

Suzuki describes detecting the movement of the image detector between two consecutive 
images. Suzuki suggests determining a common image area between a prior image and a later 
consecutive image and recording the image area of the later image which corresponds to the 
determined common image area. 

By way of example, assuming there are three images acquired from three locations X, Y 
and Z. The system of Suzuki will detect movements XY (i.e., the movement of the image 
detector between the position of X and the position of Y) and YZ (i.e., the movement of the 
image detector between the position of Y and the position of Z). Suzuki would determine a 
common image area for the X and Y images. Suzuki would record the image X and the image 
area of image Y, which corresponds to the common image area between X and Y. Afterwards, 
Suzuki would determine a common image area for the Y and Z images. The system of Suzuki 
would record the image area of image Z, corresponding to the common image area between Y 
and Z. The recorded image area of image X equals the entire image area, while the recorded 
image areas of images Y and Z are dependant on the movements XY and YZ. Additionally, 
assuming that the movement XY is different in magnitude (e.g., larger) than the movement YZ, 
the common image area between X and Y would be smaller than the common image area 
between Y and Z. Furthermore, as a result of referring only to the prior image and the movement 
between consecutive images, the recorded image area of each image can relate to different 
portions of the object. In the example set forth herein above, in the case that the movements XY 
and YZ are in the same direction, the common image area of X and Y might not include even a 
fraction of the common image area of Y and Z. Thus, the recorded area of image X and the 
recorded area of image Z might relate to different portions of the object and might not overlap. 
Of course, it is impossible to produce a stereoscopic image from non-overlapping images. 

In the system and method of the present invention, the location of origin for X, Y and Z is 

determined according to the detected movements XY and YZ. A sub-matrix of each of X, Y and 

13 



Z is selected according to the distance and direction of each of X, Y and Z from the location of 
origin. In this manner, the size of each of the sub-matrices is similar. Additionally, all sub- 
matrices relate to similar portions of the object (i.e., all sub-matrices substantially over-lap). 

The combination of Takahashi in view of Suzuki and other cited references 

A combined system of Takahashi and Suzuki would include the stereoscopic endoscope of 
Takahashi and the fluctuation compensation system of Suzuki. The combined system would 
compensate for movement of the image detector of the stereoscopic endoscope between two 
consecutive images. The compensated recorded images would possibly correspond to different 
sizes of image area and to different portions of the imaged object. Therefore, the stereoscopic 
image might be blurred (e.g., the left image is of a first portion of the object and the right image 
is of a different portion, such that the difference between the images differs from the horizontal 
disparity required for producing stereoscopic images). 

In particular, none of the cited prior art references describes detecting the movements of 
the image sensor assembly, determining an average of these movements and determining a 
location of origin accordingly. No reference describes employing the location of origin as a 
pivot for the detected images such that the center point of each sub-matrix of each image is 
positioned at the location of origin. Additionally, no reference describes selecting the sub- 
matrices to be of similar size. Thus, the movement compensation system and method of the 
claimed invention for producing stereoscopic endoscopic images patentably distinguishes over 
each of the prior art references and each of the combinations of the prior art references. 
Applicants assert that the rejections under 35 U.S. C. § 103(a) are overcome and should be 
withdrawn. 

IV. Conclusion 

As shown above, the present application is not disclosed by any of the cited prior art 
reference or any combination of prior art references. In view of the above discussion and as the 
pending claims are clearly not obvious, allowance is respectfully requested. 
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A speedy and favorable action in the form of a Notice of Allowance is hereby solicited. 
If the Examiner feels that a telephone interview may be helpful in this matter, please contact 
Applicants' representative at (612) 336-4728. 

Please consider this a PETITION FOR EXTENSION OF TIME for a sufficient number 
of months to enter these papers or any future reply, if appropriate. Please charge any additional 
fees or credit overpayment to Deposit Account No. 13-2725. 
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Respectfully submitted, 



MERCHANT & GOULD P.C. 



Dated: 




By: 




Gregory" A. Sebald 
i£g?No. 33,280 
GAS/krn 
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